Papers with intention understanding
Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLMs (2024.acl-short)
Copied to clipboard
| Challenge: | Existing work on discriminative evaluations of large language models has focused on discrimination, but this paper examines their intention understanding by examining their responses to non-literal utterances. |
| Approach: | They propose a framework to evaluate large language models’ intention understanding by examining their responses to non-literal utterances. |
| Outcome: | The proposed framework compares large language models' responses to human-like expectations and provides nuanced evaluations of their intention understanding. |
Towards Intention Understanding in Suicidal Risk Assessment with Natural Language Processing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Suicide is a global problem, with one suicide case for every 100 deaths worldwide . social networking sites are an essential forum for communication and information sharing . |
| Approach: | This paper compares natural language processing to suicidal ideation detection and risk assessment . it urges better intention understanding for reliable suicide risk assessment with computational methods . |
| Outcome: | This paper compares the performance of natural language processing to suicidal ideation detection and risk assessment tasks. |
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated the potential to mimic human social intelligence, but most studies focus on static self-report or performance-based tests. |
| Approach: | They propose a framework to assess LLMs' ability to understand and manage intentions by mapping their ability to infer the intentions of others in a game setting. |
| Outcome: | The proposed framework assesses LLMs' ability to understand and manage intentions in a game setting. |
SWE-QA: Can Language Models Answer Repository-level Code Questions? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for understanding and reasoning about entire soft-ware repositories focus on small, self-contained code snippets. |
| Approach: | They propose a repository-level code question answering benchmark to facilitate research on automated QA systems in real-world repositories. |
| Outcome: | The proposed benchmarks are designed to facilitate research on automated QA systems in real-world repositories. |